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(57) ABSTRACT 

A system and method for scaling an MPEG-2 video decoder 
subject to a resource constraint. A decoder system is pro- 
vided for decoding compressed video data arranged in a 
group of pictures, wherein the group of pictures include I 
pictures, P pictures and B pictures. The system comprises a 
processing path for decoding an error residual associated 
with the group of pictures, and a filtering system for pre- 
venting the error residual associated with B pictures from 
being decoded by the processing path. The decoder system 
may further comprise a system for selectively enabling the 
filtering system for B pictures having a DC coefficient that 
is below a predetermined threshold. 

18 Claims, 3 Drawing Sheets 
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SCALABLE MPEG-2 VIDEO DECODER 

BACKGROUND OF THE INVENTION 

1. Technical Field 

The present invention relates to decoding compressed 
video signals, and more particularly relates to a system and 
method for scaling an MPEG-2 video decoder. 

2. Related Art 

The MPEG standards are an evolving set of standards for 
video and audio compression developed by the Moving 
Picture Experts Group (MPEG). MPEG-1 was designed for 
coding progressive video at a transmission rate of about 1.5 
million bits per second. It was designed specifically for 
Video-CD and CD-i media. MPEG-2 was designed for 
coding interlaced images at transmission rates above 4 
million bits per second. The MPEG-2 standard is used for 
various applications, such as digital television (DTV) 
broadcasts, digital versatile disk (DVD) technology, and 
video storage systems. 

According to the MPEG-2 standard, an MPEG-2 
sequence is divided into a series of GOPs (Group Of 
Pictures). There are three different types of pictures, with 
each being essentially a frame of pixels. Each GOP begins 
with an Intra-coded picture (I picture) followed by an 
arrangement of forward Predictive-coded pictures (P 
pictures) and Bi-directionally predictive-coded pictures (B 
pictures). I pictures are fields or frames coded as a stand- 
alone still image. P pictures are fields or frames coded 
relative to the nearest I or P picture, resulting in forward 
prediction processing. P pictures allow more compression 
than 1 pictures through the use of motion compensation, and 
also serve as a reference for B pictures and future P pictures. 
B pictures arc fields or frames that use the most proximate 
past and future I and P pictures as a reference, resulting in 
bi-directional prediction. 

As the digital TV market gradually begins to dominate the 
TV market and other video applications become more 
desirable, the demand for systems having advanced capa- 
bilities for processing MPEG-2 pictures becomes stronger 
and stronger. The current emerging architecture for process- 
ing MPEG-2 data, such as that found in DTV set-top boxes 
and high end digital TV's, typically utilize a combination of 
a digital signal processing central processing units 
(DSPCPU), control processors, coprocessors, and software 
applications. Unfortunately, even with all these resources, 
advanced audio/visual processing functions tend to consume 
more computational power than is often available. 

One of the key elements in MPEG-2 processing is the 
MPEG-2 decoder, which converts a bitstream of compressed 
MPEG-2 data into pixel images. The main components of a 
generic MPEG-2 decoder 10 are shown in FIG. 1. There are 
four functional blocks: a variable length decoder (VLD) 12, 
an inverse quantization (IQ) system 14, an inverse discrete 
cosine transform system (IDCT) 16, and a motion compen- 
sation (MC) system 18. Memory 20 is used to store the 
reference frames. The adder combines the error residuals 
output from IDCT 16 (path 1) with motion compensation 
results (path 2) to form the final video output 24. 
Unfortunately, each of these functional components con- 
sume a significant amount of computational power, which 
drives up the cost, and limits the flexibility of digital video 
systems using MPEG-2 technology. Accordingly, making a 
highly efficient, cost effective decoder remains one of the 
main goals of all electronics manufacturers. 

One solution for addressing the processing requirements 
of MPEG-2 decoders is to provide specialized hardware 
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systems that increase computational power. For example, 
U.S. Pat. No. 5,903,311, issued to Ozcelik et al. on May 11, 
1999, which is hereby incorporated by reference, describes 
a chip that includes specialized circuits for an MPEG-2 
decoder. Unfortunately, while overall hardware costs con- 
tinue to decrease, the costs involved in designing and 
building specialized hardware such as this increase the 
expense of the decoder. 
A preferred solution therefore is to implement as much 

( functionality as possible in software, which provides sig- 
nificant cost and flexibility advantages over hardware solu- 
tions. In particular, software solutions reduce the need for 
expensive hardware, such as coprocessors, and will allow 
multiple video functions to run concurrently on a DSPCPU 
core. However, software applications tend to run loo slow to 

1 handle occasions when computationally intensive decoding 
operations are required. Accordingly, a need exists to pro- 
vide enhanced systems that will provide inexpensive 
MPEG-2 decoder solutions while maintaining an acceptable 
level of video quality. 

1 SUMMARY OF THE INVENTION 

This invention overcomes the above-mentioned 
problems, as well as others, by providing a scalable decoder 
system. In a first aspect, the invention includes a decoder 

. system for decoding a compressed video signal having B 
pictures, comprising: a first processing path that decodes an 
error residual of the compressed video signal; a second 
processing path that decodes motion compensation of the 
compressed video signal; and a filtering system for prevent- 
ing the error residual associated with B pictures from being 

1 decoded by the first processing path. The decoder system 
may further comprise a system for selectively enabling the 
filtering system. 

In a second aspect, the invention provides a method for 
decoding compressed video data having different types of 
coded pictures and outputting displayable pixel data, com- 
prising the steps of: receiving the compressed video data; 
providing a first and second processing path for processing 
the compressed video data; identifying pictures of a prede- 
termined type from the compressed video data; and prevent- 
ing a block associated with at least one of the identified 
pictures from being processed by the first processing path. 
The method may be further refined by preventing the 
processing of only those identified pictures that have a 
picture characteristic value that is below a predetermined 
threshold. 

In a third aspect, the invention provides a decoder system 
for decoding compressed video data arranged in a group of 
pictures, wherein the group of pictures includes at least one 
B picture, comprising: a processing system for decoding an 
error residual associated with the group of pictures; and a 
filtering system for preventing the error residual associated 
with the B picture from being decoded by the processing 
system. The decoder system may further include a system 
for selectively enabling the filtering system. 

Accordingly, it is an advantage of the present invention to 
provide a video processing solution that results in an accept- 
able level of output degradation in a real-time system that is 
subject to resource constraints. 

It is a further advantage to provide a system and method 
for efficiently scaling a decoder while maintaining an 
acceptable level of output quality. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The preferred exemplary embodiment of the present 
invention will hereinafter be described in conjunction with 
the appended drawings, where like designations denote like 
elements, and: 
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FIG. 1 depicts a functional diagram of a video decoder; provides a mechanism that inhibits B picture data from 

FIG. 2 depicts a graphical representation of a group of being processed by the first processing path. Although 

pictures (GOP); and shown as residing as part of VLD 26, it is understood that 

FIG. 3 depicts a functional diagram of a video decoder filtering system 28 can be implemented anywhere within the 

having a filtering system in accordance with a preferred 5 first processing path. Moreover, it is understood that decoder 

embodiment of the present invention. 11 aod filtering system 28 can be implemented in hardware, 

software, or a combination of hardware and software. The 

DETAILED DESCRIPTION OF THE DRAWINGS decoder 11 and filtering system 28 may be implemented by 

any type of computer system — or other apparatus adapted 

Overview io f or carrying out the methods described herein. A typical 

As noted above, in the MPEG-2 standard, there are three combination of hardware and software could be a general- 
types of pictures: I, P and B, and the pictures are coded purpose computer system with a computer program that, 
within a unit referred to as a group of pictures (GOP). when loaded and executed, controls the computer system 
Referring to FIG. 2, a GOP is depicted containing a series of such ^at it carries out the methods described herein, 
pictures beginning with I pictures. As can be seen by the 15 Alternatively, a specific use computer, containing special- 
arrows above the GOP, the decoding of I pictures do not hardware for carrying out one or more of the functional 
need any previous decoded picture. Therefore, there is no tasks of invention could be utilized. The present inven- 
prediction drift. In contrast, the decoding of a P picture tion caD ^ be embedded in a computer program product, 
depends on the previously decoded I or P picture. If the which comp^es ^ ^ features enabling the impiementa- 
prediction is not accurate, the prediction drift accumulates. 20 tion of the methods and functions described herein, and 
B pictures are predicted from previous and future I or P which— when loaded in a computer system— is able to carry 
pictures. Both P and B pictures carry prediction errors, but out thesc methods and functions. Computer program, soft- 
B pictures do not contribute to the error accumulation and ware program, program, program product, or software, in 
propagation. tce P resent context mean any expression, in any language, 

tv i«- p j j tj > . r c . . v . 25 code or notation, of a set of instructions intended to cause a 

The quality of I and P pictures (i.e.. reference pictures) is , . - f , 

J j - A. * * <• ■ *. system having an information processing capability to per- 

cntical because any error made in these two types o t pictures J . , „ - , . *> • . 

* . .l • ti r» form a particular function either directly or after either or 

will propagate to the following pictures. However, a B . iL £ \, - M . . . 7 , . 

^ K 4 * -u * » i both of the following: (a) conversion to another language, 

picture does not contribute to any error propagation and , . ,; «_v * ■ - 

f. f u • . i-* c A. c u coc *e or notation; and/or (b) reproduction in a different 

therefore has no impact on the quality of the following „ . . - ' v 7 v 

ri ,t r * . ■ c n • • . 30 material form, 
pictures. Furthermore, the computation of B pictures is the 

most expensive one of all three types of pictures due to the 1- Total Filtering of the Residual Computation Of B Pictures 

bi-directional prediction. Accordingly, the potential savings Pictures are comprised of blocks of data relating to the 

of scaling back the computation of B pictures is not only picture. One type of block is an error residual block that 

greater than the other picture types, but will also result in 35 includes DCT (discrete cosine transform) coefficients, 

less overall quality degradation. Experimentation has shown that for slow motion pictures, 

Referring back to FIG. 1, from the MPEG-2 decoding the error residual blocks have very small values. Completely 

system point of view, I pictures require decoding only along skipping the decoding of such error residual blocks will 

the first processing path, (PATH 1) which decodes coded therefore not degrade the output quality severely, 

pixel data. For decoding P and B pictures, two processing 4Q Nonetheless, the computational resources that are saved by 

paths exist. In particular, the first processing path (PATH 1) skipping this step are considerable. Accordingly, for certain 

decodes the coded prediction error residual and a second types of video data, implementing a scheme in which the 

processing path (PATH 2) decodes the motion compensation error residual associated with B pictures is skipped will yield 

results. The combination of these two paths gives the final an acceptable output and provide significant computational 

decoded pixel output. Of all the functional blocks, the IDCT 45 savings. Experimental results show that this method works 

block 16 is one of the most computationally intensive units. particularly well for slow to moderate motion sequence. 

Accordingly, scaling back processing performed along the As noted, filtering system 28 shown in FIG. 3 provides the 

first processing path will generally result in the greatest mechanism for implementing such a scheme. At VLD 26, a 

computational savings. determination is made by filtering system 28 to determine if 

50 the current picture is a B picture. A preferred method for 

Exemplary Embodiments determining this is to examine the picture header, which will 

The present invention seeks to capitalize on the potential indicate the type of picture (i.e., I, P or B). If the current 

computational savings identified above by scaling back: (1) picture is a B picture, then the error residual information will 

B picture decoding, and (2) processing performed by the first not be sent to IQ 14 and IDCT 16, thereby causing the first 

processing path in an MPEG-2 decoder. A first scheme 55 processing path to be skipped. The decoding process will 

proposes to skip or prevent the decoding of error residual only proceed along the second processing path, 

associated with B pictures. This process is referred to herein The computational savings for skipping the processing of 

as filtering. A second scheme proposes to selectively skip or B picture residual error can be quite significant. One way to 

prevent (i.e., filter) me decoding of error residual associated estimate the savings is to analyze the complexity level 

with B pictures. Referring to FIG. 3, a modified MPEG-2 50 required to process a GOP for a decoder with and without 

decoder 11 is shown. Decoder 11 includes the same func- filtering system 28. The complexity level for processing a 

tionality as decoder 10 shown in FIG. 1, except that decoder GOP can be estimated as follows, where N is the size of the 

11 includes a filtering system 28, and may further comprise GOP and M is the spacing between reference frames, 

an enabling system 30 for selectively enabling filtering Assume that C^, C fQ+fDCr , C MC and C + represent the 

system 28. 65 needed average compute power (complexity level) for each 

As noted above, the decoding of error residual occurs functional block VLD, IQ+IDCT, MC and the adder "+," 

along the first processing path (PATH 1). Filtering system 28 respectively, for each picture or frame. Then the average 
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compute power for a generic MPEG-2 decoder (FIG. 1) in 
one GOP, denoted as CL, is estimated as 



Cm 



CvLD ' Nl+P+B + ClQ+IDCT • NuP+B + Cue ■ Np+B + Coddrr • NfKB 
N^P+B 



Note that N„ Np, and N B denotes the total number of each 
type of frame within the GOP, denotes the total 

number of P and B frames within one GOP, and ti UP ^S. 

By further assuming that CS is the average compute 
power of the MPEG-2 decoder (FIG. 3) in one GOP that 
skips the residual calculation for B frames. Then 



block is a high-energy block, then the error residual 
block is processed as if it were a generic decoder 10 
(PATH 1 and PATH 2 are implemented). 
If the DC coefficient is smaller than the preset threshold, 
the residual computation for this block is skipped (only 
the second processing path is used for decoding), and 
the video output is equal to the MC result. 
If it is assumed that some ratio "y" of blocks in B pictures 
are kept in the decoding process (where 0 [f[l), then the 
complexity of this scheme can be estimated as follows: 



c, = l- 



(1 -jXCiQ+iDcr + CvUt,). 



1 Ni+p+B 

where N /4 ^ denotes the total number of I and P pictures 
within one GOP. Then, 

' CvLD- #/+P+S + CtQ+tQCT • A// + p+fl + C MC ■ Np+B + Coetter-Np+B 

Assume C^ol, then 



Then, for example, if a GOP was defined as having size 
N=15 and M=3, then N B =10, and 

Assume that C IQ + IDC1 +C adder represents 30% of the com- 
plexity of the decoder (which is a reasonable assumption), 
then C f =80%. This indicates that if all the residual compu- 
tations for B pictures in the MPEG-2 decoder 11 are skipped 
for the denned GOP, then the total compute complexity per 
frame is 80% of the original generic decoding complexity. 
2. Selective Filtering of the Residual Computation Of B 
Pictures 

As mentioned earlier, the first scheme works well for 
moderate to slow or no motion sequence. For fast motion 
sequences, such as sports scenes, the decoding quality of the 
above scheme results in a potentially unsatisfactory output. 
The following scheme, which selectively blocks the residual 
computation of B pictures, is proposed to adapt to all kinds 
of sequences, regardless of the motion. In the embodiment 
depicted in FIG. 3, VLD 26 is shown with filtering system 
28, which includes enable system 30. Enable system 30 
selectively enables filtering mechanism 28, i.e., it dynami- 
cally decides whether to allow or skip the residual compu- 
tation of B pictures along the first processing path. 

One implementation for this operation is as follows. If the 
picture header indicates that the current picture is a B 
picture, a decision is made to either skip the processing 
along the first processing path, or proceed with the normal 
decoding steps (first path+second path). The decision may 
be based on any relevant criteria. A preferred method is to 
compare the DC coefficient of each block with a preset 
threshold. The DC coefficient is the coefficient generally 
located at the top left comer of the DCT block and is a good 
indicator of the energy in the error residual block. 

The procedure is described as follows: 

If the DC coefficient of current block is greater than or 
equal to the preset threshold, indicating the current 



15 This equation indicates that when the number of B pictures 
(Nj) increases, particularly the low energy blocks in B 
pictures (7 small), the average complexity level of the 
decoder decreases. Experimental results show that in a 
typical fast motion scene, 80%-90% of the B blocks are 

20 considered to be low energy blocks, and are therefore 
skipped as in the first scheme. 

The foregoing description of the preferred embodiments 
of the invention have been presented for purposes of illus- 
tration and description. They are not intended to be exhaus- 

25 five or to limit the invention to the precise form disclosed, 
and obviously many modifications and variations are pos- 
sible in light of the above teachings. Such modifications and 
variations that are apparent to a person skilled in the art are 
intended to be included within the scope of this invention as 
defined by the accompanying claims. For instance, although 

30 the description provided herein generally relates to MPEG-2 
decoders, it is understood that the invention can be applied 
to any system where different types of pictures are processed 
by multiple processing paths. 
We claim: 

35 1. A decoder system for decoding a compressed video 
signal having B pictures, comprising: 

a first processing path that decodes an error residual of the 

compressed video signal; 
a second processing path that decodes motion compen- 
40 sation of the compressed video signal; and 

a filtering system for preventing the error residual asso- 
ciated with B pictures from being decoded by the first 
processing path. 

2. The decoder system of claim 1, wherein the first 
45 processing path includes a variable length decoder, an 

inverse quantization system, and an inverse discrete cosine 
transform system. 

3. The decoder system of claim 2, wherein the filtering 
system resides within the variable length decoder. 

50 4. The decoder system of claim 3, wherein the filtering 
system prevents residual discrete cosine transform coeffi- 
cients from being passed to the inverse quantization system 
and the inverse discrete cosine transform system. 

5. The decoder system of claim 1, wherein the filtering 
55 system identifies B pictures by examining header details of 

each frame of the compressed video signal. 

6. The decoder system of claim 1, further comprising a 
system for selectively enabling the filtering system. 

7. The decoder system of claim 6, wherein the system for 
60 selectively enabling the filtering system compares a DC 

coefficient of each B picture with a preset threshold. 

8. The decoder system of claim 7, wherein the filtering 
system is enabled when the DC coefficient is smaller than 
the preset threshold. 

65 9. The decoder system of claim 7, wherein the filtering 
system is disabled when the DC coefficient is larger than the 
preset threshold. 
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10. A method for decoding compressed video data having 
different types of coded pictures and outputting displayablc 
pixel data, comprising the steps of: 

receiving the compressed video data; 

providing a first and second processing path for process- 
ing the compressed video data: 

identifying pictures of a predetermined type from the 
compressed video data; and 

preventing a block associated with at least one of the 
identified pictures having a DC coefficient that is below 
a predetermined threshold from being processed by the 
first processing path. 

11. The method of claim 10, comprising the further step 
of preventing blocks associated with each of the identified 
pictures from being processed by the first processing path. 

12. A method for decoding compressed video data having 
different types of coded pictures and outputting displayable 
pixel data, comprising the steps of: 

receiving the compressed video data; 

providing a first and second processing path for process- 
ing the compressed video data: 

identifying pictures of a predetermined type from the 
compressed video data; and 

preventing a block associated with at least one of the 
identified pictures from being processed by the first 
processing path; 

wherein the compressed video data is in an MPEG-2 
format, the different types of coded pictures include 1 
pictures, P pictures and B pictures, and the at least one 
identified picture is a B picture. 

13. A method for decoding compressed video data having 
different types of coded pictures and outputting displayable 
pixel data, comprising the steps of: 



receiving the compressed video data; 

providing a first and second processing path for process- 
ing the compressed video data: 

identifying pictures of a predetermined type from the 
5 compressed video data; and 

preventing a block associated with at least one of the 
identified pictures from being processed by the first 
processing path; 
io wherein the first processing path includes an inverse 
quantization system and an inverse discrete cosine 
transform system, and the second processing path 
includes a motion compensation system. 

14. A decoder system for decoding compressed video data 
15 arranged in a group of pictures, wherein the group of 

pictures include at least one B picture, comprising: 

a processing system for decoding an error residual block 

associated with the group of pictures; and 
a filtering system for preventing the error residual block 
20 associated with the B picture from being decoded by 
the processing system. 

15. The decoder system of claim 14, wherein the process- 
ing system includes an inverse quantization system and an 
inverse discrete cosine transform system. 

25 16. The decoder system of claim 14, further comprising a 
system for selectively enabling the filtering system. 

17. The decoder system of claim 16, wherein the system 
for selectively enabling the filtering system enables the 
filtering system based on information extracted from the B 

30 picture. 

18. The decoder system of claim 16, wherein the system 
for selectively enabling the filtering system enables the 
filtering system when a DC coefficient of the B picture is 
below a predetermined threshold. 

35 
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